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Abstract. The stochastic Boolean satisfiability (SSAT) problem has been introduced 
by Papadimitriou in 1985 when adding a probabilistic model of uncertainty to prepo- 
sitional satisfiability through randomized quantification. SSAT has many applications, 
among them probabilistic bounded model checking (PBMC) of symbolically represented 
Markov decision processes. This article identifies a notion of Craig interpolant for the 
SSAT framework and develops an algorithm for computing such interpolants based on a 
resolution calculus for SSAT. 

As a potential application area of this novel concept of Craig interpolation, we address 
the symbolic analysis of probabilistic systems. We first investigate the use of interpolation 
in probabilistic state reachability analysis, turning the falsification procedure employing 
PBMC into a verification technique for probabilistic safety properties. We furthermore 
propose an interpolation-based approach to probabilistic region stability, being able to 
verify that the probability of stabilizing within some region is sufficiently large. 



Introduction 

Papadimitriou |Pap85| has proposed the idea of modeling uncertainty within propositional 
satisfiability (SAT) by adding randomized quantification to the problem description. The 
resultant stochastic Boolean satisfiability (SSAT) problems consist of a quantifier prefix 
followed by a propositional formula. The quantifier prefix is an alternating sequence of ex- 
istentially quantified variables and variables bound by randomized quantifiers. The meaning 
of a randomized variable x is that x takes value true with a certain probability p and value 
false with the complementary probability 1 — p. Due to the presence of such probabilistic 
assignments, the semantics of an SSAT formula <3? no longer is qualitative in the sense that 
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is satisfiable or unsatisfiable, but rather quantitative in the sense that we are interested 
in the maximum probability of satisfaction of Intuitively, a solution of $ is a strategy 
for assigning the existential variables, i.e. a tree of assignments to the existential variables 
depending on the probabilistically determined values of preceding randomized variables, 
such that the assignments maximize the probability of satisfying the propositional formula. 

In recent years, the SSAT framework has attracted interest within the Artificial In- 
telligence community, as many problems from that area involving uncertainty have con- 
cise descriptions as SSAT problems, in particular probabilistic planning problems [LMP01, 
IML98 , ML03J. Inspired by that work, other communities have started to exploit SSAT and 
closely related formalisms within their domains. The Constraint Programming community 
is working on stochastic constraint satisfaction problems [Wal02, BSQ6] to address, among 
others, multi-objective decision making under uncertainty [BS07] ■ Recently, a technique 
for the symbolic analysis of probabilistic hybrid systems based on stochastic satisfiability 
has been suggested by the authors (FHT081 ITF091 IFTE101 ITEFllj . To this end, SSAT 
has been extended by embedded theory reasoning over arithmetic theories, as known from 
satisfiability modulo theories (SMT) [BSST09], which yields the notion of stochastic satis- 
fiability modulo theories (SSMT). By the expressive power of SSMT, bounded probabilistic 
reachability problems of uncertain hybrid systems can be phrased symbolically as SSMT 
formulae yielding the same probability of satisfaction [FHT08| ITF091 IFTE101 ITEFll j. As 
this bounded model checking approach yields valid lower bounds lb of the probability of 
reaching undesirable system states along unbounded runs, it is able to falsify probabilistic 
safety requirements of shape "a system error occurs with probability at most 0.1%o" , namely 
if a lower bound lb > 0.1%o is computed. 

Though the general SSAT problem and even its restriction to 2CNF, i.e. to formu- 
lae in conjunctive normal form containing clauses with two literals only, are PSPACE- 
complete [TF10| . the plethora of real- world applications calls for practically efficient al- 
gorithms. The first SSAT algorithm, suggested by Littman [Lit99], extends the Davis- 
Putnam-Logemann-Loveland (DPLL) procedure [DP60J IDLL62] for SAT with appropriate 
quantifier handling and algorithmic optimizations like thresholding. Majercik further im- 
proved the DPLL-based SSAT algorithm by non- chronological backtracking |Maj04|. The 
SSMT algorithm from [FHT081 ITF081 ITF091 ITEFllj being implemented in the SSMT 
tool SiSAT builds on the DPLL-based SSAT procedures plus conflict-driven clause learn- 
ing, but also integrates an underlying theory solver addressing non-linear arithmetics, and 
was successfully applied to realistic case studies featuring hybrid discrete-continuous state 
spaces [TF09J IFTE10 , TEFlT]. Unlike these explicit tree-traversal approaches and mo- 
tivated by work on resolution for propositional and first-order formulae [Rob65] and for 
quantified Boolean formulae (QBF) [BKF95], the authors have recently developed an alter- 
native SSAT procedure based on resolution [TF10] , 

In this article, we investigate the concept of Craig interpolation for SSAT. Given two 
formulae A and B for which A =^ B is true, a Craig interpolant [Cra57| X is a formula 
over variables common to A and B that "lies in between" A and B in the sense that 
A => I and I => B. In the automatic hardware and software verification communi- 
ties, Craig interpolation has found widespread use in model checking algorithms, both as 
a means of extracting reasons for non-concretizability of a counterexample obtained on 
an abstraction as well as for obtaining a symbolic description of reachable state sets. In 
McMillan's approach [McM03, McM05], interpolants are used to symbolically describe an 
overapproximation of the step-bounded reachable state set. If the sequence of interpolants 
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Figure 1: Semantics of an SSAT formula depicted as a tree. 

thus obtained stabilizes eventually, i.e. no additional state is found to be reachable, then 
the corresponding state-set predicate R has all reachable system states as its models. The 
safety property that states satisfying B, where B is a predicate, are never reachable is then 
verified by checking R A B for unsatisfiability. 

Given McMillan's verification approach to reachability analysis of non-probabilistic sys- 
tems based on Craig interpolation for SAT, it is natural to ask whether a corresponding 
probabilistic counterpart can be developed, i.e. a verification approach to probabilistic reach- 
ability analysis of probabilistic systems based on Craig interpolation for stochastic SAT. Such 
an approach would complement the aforementioned falsification procedure for probabilistic 
systems based on SSAT/SSMT. In this article, we suggest a solution to the issue above. 

In addition to probabilistic state reachability, we address the problem of probabilistic 
region stability. The latter problem is motivated by the notion of region stability for non- 
probabilistic hybrid systems |PW07al IP W07b] . where a system is called stable with respect 
to some region R iff all system runs eventually reach R and finally stay in R forever. In 
this article, we suggest an adaptation of region stability to the probabilistic case along 
with a symbolic, interpolation-based procedure for the verification of probabilistic stability 
properties like "the probability that the system stabilizes within region R is at least 99.9%". 

Structure of the article. After a formal introduction to SSAT in Section [H Section [2] is de- 
voted to a generalization of the notion of Craig interpolants suitable for SSAT. Thereafter, 
Section [3] elaborates on an algorithm for computing such generalized Craig interpolants, 
which relies on a resolution calculus for SSAT. The application of generalized Craig in- 
terpolation to the symbolic analysis of probabilistic systems, namely to probabilistic state 
reachability as well as to probabilistic region stability, is then addressed in Section [H where 
applicability of these novel techniques is illustrated on small examples. Section [5] finally 
concludes the article. 

1. Stochastic Boolean satisfiability 

A stochastic Boolean satisfiability (SSAT) formula is of the form $ = Q : (p with a prefix 
Q = Q lXl ...Q n x n of quantified propositional variables Xj, where Qi is either an existential 
quantifier 3 or a randomized quantifier H Pi with a rational constant < pi < 1, and a 
propositional formula tp such that Var(ip) C {xi, . . . , x n }, where Var{ip) denotes the set 
of all (necessarily free) variables occurring in ip. Note that SSAT formula $ thus has 
no free variables. Without loss of generality, we assume that <p is in conjunctive normal 
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form (CNF), i.e. a conjunction of disjunctions of propositional literals. A literal I is a 
propositional variable, i.e. I = Xj, or its negation, i.e. t = -iXj. A clause is a disjunction 
of literals. Throughout the article and without loss of generality, we require that a clause 
does not contain the same literal more than once as I V £ = I. Consequently, we may also 
identify a clause with its set of literals. The semantics of <£, as illustrated in Figure Q3 is 
defined by the maximum probability of satisfaction Pr(&) as follows. 

p , s f if p is logically equivalent to false 

^ ' 1 1 if 99 is logically equivalent to true 

Pr(3x Q: if) = max(Pr(Q : ip [true/x] ), Pr(Q : <p[f alse/x])) 
Pr(U p x Q-p>)=p- Pr(Q : ^[true/x]) + (1 - p) ■ Pr(Q : ip[f alse/x]) 

Note that the semantics is well-defined as $ has no free variables such that all variables 
have been substituted by the constants true and false when reaching the quantifier-free 
base case. 



2. Generalized Craig interpolants 

Craig interpolation |Cra57] is a well-studied notion in formal logics which has several ap- 
plications in Computer Science, among them model checking [McM03, McM05j. Given two 
formulae p and ip such that ip =>■ ip is valid, a Craig interpolant for ((p, ip) is a formula I 
which refers only to common variables of (p and ip, and I is "intermediate" in the sense that 
tp =>• I and I =/- ip. Such interpolants do trivially exist in all logics permitting quantifier 
elimination, for instance, in propositional logic. The observation that p => ip holds iff 
ip A -rip is unsatisfiable gives rise to an equivalent definition which we refer to in the rest of 
the articled given an unsatisfiable formula p A -up, a formula X is a Craig interpolant for 
{ip, ip) iff both p A —CL and X A —iip are unsatisfiable and I mentions only common variables. 

In this section, we investigate the issue of Craig interpolation for stochastic SAT. We 
propose a generalization of Craig interpolants suitable for SSAT and show the general 
existence of such interpolants. In Section [3j we then devote our attention to an automatic 
method for computing generalized Craig interpolants based on a resolution calculus for 
SSAT. 

When approaching a reasonable definition of interpolants for SSAT, the semantics of 
the non-classical quantifier prefix poses problems: Let $ = Q : (A AB) be an SSAT formula. 
Each variable in A A B is bound by Q, which provides the probabilistic interpretation of 
the variables that is lacking without the quantifier prefix. This issue can be addressed 
by considering the quantifier prefix Q as the global setting that serves to interpret the 
quantifier- free part, and consequently interpreting the interpolant also within the scope of 
Q, thus reasoning about Q : (AA^I) and Q : (IAB). A more fundamental problem is that a 
classical Craig interpolant for $ only exists if Pr(&) = 0, since AAB has to be unsatisfiable 
by definition of a Craig interpolant which applies iff Pr(Q : (AAB)) = 0. The precondition 
that Pr(Q : (AAB)) = would be far too restrictive for application of interpolation, as the 
notion of unsatisfiability of A A B is naturally generalized to satisfiability with insufficient 
probability, i.e. Pr(Q : (A A B)) being "sufficiently small", in the stochastic setting. Such 

^This is of technical nature as SSAT formulae are interpreted by the maximum probability of satisfaction. 
As the maximum probability that an implication tp => holds is inappropriate for our purpose, we reason 
about the maximum satisfaction probability p of the negated implication, i.e. of <p A instead. The latter 
relates to the minimum probability 1 — p that ip ip holds, which is the desired notion. 
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relaxed requirements actually appear in practice, for instance, in probabilistic verification 
where safety properties like "a fatal system error is never reachable" are frequently replaced 
by probabilistic ones like "a fatal system error is reachable only with (sufficiently small) 
probability of at most 0.1%o". Motivated by above facts, interpolants for SSAT should also 
exist when A A B is satisfiable with reasonably low probability. 

The resulting notion of interpolation, which is to be made precise in Definition 12.11 
matches the following intuition. In classical Craig interpolation, when performed in logics 
permitting quantifier elimination, the Craig interpolants of (A, —>B) form a lattice with im- 
plication as its ordering, A 3 = 3ai, . . . a a : A as its bottom element and B = ->3bi, ...bp:B 
as its top element, where the a% and b% are the local variables of A and of B, respectively. 
In the generalized setting required for SSAT0, A =>- -if? and thus A 3 =>■ B may no longer 
hold such that the above lattice can collapse to the empty set. To preserve the overall 
structure, it is however natural to use the lattice of propositional formulae "in between" 
A 3 A B as bottom element and A 3 V B as top element instead. This lattice is non-empty 
and coincides with the classical one whenever A A B is unsatisfiable. 

Definition 2.1 (Generalized Craig interpolant). Let A, B be propositional formulae and 



V A := Var(A) \ Var(B) = {a x ,...,a a }, V B := Var(B) \ Var(A) = {bx,...,bp}, V A>B := 



Var(A)n Var(B), A 3 = 3a±, . . . , a a : A, and B = ->3bi, . . . , bp : B. A propositional formula 



X is called generalized Craig interpolant for (A,B) iff Var(X) C Va,Bi (A 3 AB ) => X, 



Given any two propositional formulae A and B, the four quantifier-free propositional 
formulae equivalent to A 3 A B , to A 3 , to B , and to A 3 V B , are generalized Craig 
interpolants for (A,B). These generalized interpolants always exist since propositional 
logic has quantifier elimination. 

While Definition 12 . 1 1 motivates the generalized notion of Craig interpolant from a model- 
theoretic perspective, we state an equivalent definition of generalized Craig interpolants in 
Lemma 12.21 that substantiates the intuition of generalized interpolants and allows for an 
illustration of their geometric shape. Given two formulae A and B, the idea of generalized 
Craig interpolant is depicted in Figure [2j The set of solutions of A is defined by the rectangle 
on the Va, V^s-plane with a cylindrical extension in Ve-direction as A does not contain 
variables in Vb- Similarly, the solution set of B is given by the triangle on the Vb,Va,b- 
plane and its cylinder in T/4-direction. The solution set of A A B is then determined by the 
intersection of both cylinders. Since AABA^(AAB) is unsatisfiable, the sets A A ~>(A A B) 
and B A ~^{A A B) are disjoint. This gives us the possibility to talk about interpolants wrt. 
these sets. However, a formula X over only common variables in Va,b may not exist when 
demanding A A -<(A A B) A -X and X A B A ->{A A B) to be unsatisfiable. This is indicated 
by Figure [2] and proven by the simple example A = (a), B = (b). As Va,b = 0, X is either 
true or false. In first case, true A (b) A -i(sA6) is satisfiable, while (a) A ->(a A b) A ->f alse 
is in second case. If we however project the solution set of A A B onto the V^^-axis and 
subtract the resulting hyperplane tS^s from A and B then such a formula X over Va,b~ 
variables exists. The next lemma formalizes such generalized interpolants X and shows their 
equivalence to the ones from Definition 12.11 

2 Though the concept seems to be more general, this article addresses SSAT only. 
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Figure 2: Geometric interpretation of a generalized Craig interpolant X. Va-, Vb~, and 
V/l,s-axes denote assignments of variables occurring only in A, only in B, and in 
both A and B, respectively. 

Lemma 2.2 (Generalized Craig interpolant for SSAT). Let $ = Q : (A A B) be some 
S 'SAT formula, Va, Vb, Va,b be defined as in Definition \2.1\ and Sa,b be a propositional 
formula with Var{SA,B) ^ Va,b such that Sa,b = 3oi, . . . , a a , b%, . . . , bp : (A A B). Then, 
a propositional formula T is a generalized Craig interpolant for (A, B) iff the following 
properties are satisfied. 

(1) Var(l)QV A . B 

(2) Pr(Q : (A A ^S A ,B A -<T)) = 

(3) Pr(Q: (1 A B A^S a ,b)) =0 

Proof. As Var(I) C Va,b holds for generalized Craig interpolants X, it remains to show 
that (A 3 AB V ) => 1 and X (A 3 V B V ) iff Pr(Q : (A A ^5 a ,b A -CT)) = and Pr(Q : 
(XABA^S a ,b)) = 0. Observe that |= (i 3 Afi V ) 2" iff |= Vai, . . . , a a : (AA5 V ) ^ X iff 
K^AB V ) => X iff |= (A A (-^ 3 V #)) =► X iff (= (t! A ^S a ,b) => lift A A ^S a ,b A -X 
is unsatisfiable iff Pr(Q : (A A ~^S a ,b A ~>X)) = 0. Analogously, \= X => (,4 3 V £ V ) iff 
|= V&i, . . . , bp : X (,4 3 V -.B) iff |= X => (A 3 V -.S) iff |= X => ((A 3 A ^B V ) V -.5) iff 
|= X (5^,5 V nfl) iff X A -<Sa,b A B is unsatisfiable iff Pr(Q : (X A B A ->Sa,b)) = 0. □ 
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We remark that the concept of generalized Craig interpolants is a generalization of Craig 
interpolants in the sense that whenever AAB is unsatisfiable, i.e. when Pr(Q : (AAB)) = 0, 
then each generalized Craig interpolant I for (A, B) actually is a Craig interpolant for A 
and B since Sa,b = false. 



In this section, we proceed to the efficient computation of generalized Craig interpolants. 
The remark following Definition 12.11 shows that generalized interpolants can in principle 
be computed by explicit quantifier elimination methods, like Shannon's expansion or bi- 
nary decision diagrams (BDDs). We aim at a more efficient method based on SSAT 
resolution [TF10] akin to resolution-based Craig interpolation for propositional SAT by 
Pudlak |Pud97] . The latter approach has been integrated into DPLL-based SAT solvers 
featuring conflict analysis and successfully applied to symbolic model checking [McM03, 
IMcM05| . To this end, we first recall the sound and complete resolution calculus for SSAT 
from |TF10j in Section 13.11 Thereafter, SSAT resolution is enhanced in order to compute 
generalized Craig interpolants in Section [3.21 

3.1. Resolution for SSAT. As basis of the SSAT interpolation procedure introduced in 
Section 13.21 we recall the sound and complete resolution calculus for SSAT from [TF10] . 
subsequently called S-resolution. In contrast to SSAT algorithms implementing a DPLL- 
based backtracking procedure, thereby explicitly traversing the tree given by the quantifier 
prefix and recursively computing the individual satisfaction probabilities for each subtree 
by the scheme illustrated in Figure [IJ S-resolution follows the idea of resolution for propo- 
sitional and first-order formulae [Rob65 and for QBF formulae [BKF95] by deriving new 
clauses c p annotated with probabilities < p < 1. S-resolution differs from non-stochastic 
resolution, as such derived clauses c p need not be implications of the given formula, but 
are just entailed with some probability. Informally speaking, the derivation of a clause c p 
means that under SSAT formula Q : ip, the clause c is violated with a maximum probability 
at most p, i.e. the satisfaction probability of Q : (<p A —>c) is at most p. More intuitively, the 
minimum probability that clause c is implied by ip is at least 1 — p@ Once an annotated 
empty clause P is derived, it follows that the probability of the given SSAT formula is at 
most p, i.e. Pr(Q : (ip A -if alse)) = Pr(Q : ip) < p. 

In what follows, let Q : <p be an SSAT formula with ip in CNF. Without loss of generality, 
ip contains only non-tautological clauses^, i.e. Vc G (p : ty= c. Let Q = Q\x± . . . Q n x n be the 
quantifier prefix and tp be some propositional formula with Var((p) C {x\, . . . ,x n }. The 
quantifier prefix Q((p) is defined to be shortest prefix of Q that contains all variables from 
ip, i.e. Q(ip) = Q\X\ . . .QiXi where Xj £ Var(ip) and for each j > i : xj ^ Var(ip). Let 
further be Var(tp) i-k-= {x\, . . . ,Xk} for each integer < k < n. For a non-tautological 
clause c, i.e. if ^= c, we define the unique assignment ff c that falsifies c as the mapping 



Consequently, c evaluates to false under assignment ff c . 

^We remark that Pr(Q : tp) — 1 — Pr(Q' : -^ip), where Q 1 arises from Q by replacing existential quantifiers 
by universal ones, where universal quantifiers call for minimizing the satisfaction probability. 
Tautological clauses c, i.e. |= c, are redundant, i.e. Pr(Q : (ip A c)) = Pr(Q : ip). 
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Starting with clauses in <p, S-resolution is given by the consecutive application of 
rules IR.1 1 to IR.3I to derive new clauses c p with < p < 1. Rule IR.1 1 derives a clause 
c° from an original clause c in ip. Referring to the definition of Pr(Q : ip) in Section [TJ 
IR.1 1 corresponds to the quantifier-free base case where ip is equivalent to false under any 
assignment that falsifies c. 

c £ ip 



(R.1 



Similarly. IR.2l reflects the quantifier-free base case in which ip is equivalent to true under any 
assignment t' that is conform to the partial assignment r since |= ip[r(xi) / 'x\\ . . . [r(xj)/xj]. 
The constructed clause c 1 then encodes the opposite of this satisfying (partial) assignment 
r. We remark that finding such a r in the premise of IR.2I is NP-hard (equivalent to finding 
a solution of a propositional formula in CNF). This strong condition on r is not essential 
for soundness and completeness and could be removed^] but, as mentioned above, facilitates 
a less technical presentation of generalized interpolation in Section 13.21 Another argument 
justifying the strong premise of IR.2I is a potential integration of S-resolution into DPLL- 
based SSAT solvers since whenever a satisfying (partial) assignment r of ip is found by an 
SSAT solver then r meets the requirements of IR.21 

c C {x, ->x\x G Var((p)}, ^= c, Q(c) = Q\X\ . . . QiXi, 
for each r : Var{tp) > B with \/x G Var{c) : t(x) = ff c {x) : 

\= ip[T(xi)/xi] . . . [T(Xi)/Xi] 



c 1 



(R.2) 



P 



Rule IR.3I finally constitutes the actual resolution rule as known from the non-stochastic 
case. Depending on whether an existential or a randomized variable is resolved upon, the 
probability value of the resolvent clause is computed according to the semantics Pr(Q : ip) 
defined in Section [TJ 

(ci y^x)P\(c 2 Vx)P 2 ,Qx €Q,Qx£ Q(ci Vc 2 ),^ (ci Vc 2 ), 
max(pi,p 2 ) ; Q = 3 

p x -Pi + (l- Px )-P2 ;Q = U Px 

(ciVc 2 )p 1 ' ' 

The derivation of a clause c p by IR.1 1 from c, by IR.2[ and by IR.3I from c^ 1 , Cg 2 is denoted 
by c h[Rj] c p , by K [r^] c p , and by (c^ 1 ,^ 2 ) I~ [r3] c p , respectively. Given rules IR.1 1 to IR.3I 
S-resolution is sound and complete in the following sense. 

Lemma 3.1. Let clause c p be derivable by S-resolution and let Q(c) = Q±x\ . . . QiX^. For 
each t : Var(ip) > IB with \/x G Var(c) : t(x) = ff c (x) it holds that Pr{Qi + \Xi + i . . . Q n x n '■ 
99[r(xi)/xi] . . . [T(xi)/xi]) = p. 

Proof. We show the lemma by induction over the application of rules IR.1 [ IR.2[ and IR.3I 
The base case is given by rules IR.11 and IR.21 By construction of r, <^[t(xi)/xi] . . . [r(xj)/xj] 
is unsatisfiable for IR.1 l and tautological for IR .2! which immediately establishes the result for 
the base case. Now assume that the assumption holds for all clauses in the premises of IR.31 
i.e. 

Pr(Q j+ ix j+ i . . . Q n x n : ^{t^xi) / x{\ . . . [Tiixj^/xj-i^true/xj]) = pi, 
Pr(Q j+ ix j+ i . . . Q n x n : (p[T 2 (xi) / x x ] . . . [r 2 (a; :/ _i)/a;j_i][f alse/xj]) = p 2 , 



^Then, Lemma |3 . 1 1 must be weakened to Pr(Qi+\Xi+i . . .Q n x n : ip[r(xi) / 'x{\ . . . [r{xi) / Xi\) < p, as for 
one inal S-resolution [TF10| . 
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where Xj = x with j > i+1. By definition of Pr, for each r with t(x) = ti(x) if x 6 Var(c\) 
and t{x) = T2(x) if x 6 Var{cq) we then have 

Pr(QjXj Q j+ ix j+1 . . . Q n x n : ip[r (xi)/xi\ . . . [r(xj-i)/xj-i\) = p. 

The result is obvious for j = i + For j > i + 1, note that variables Xj+i, . . . , Xj_i do not 
occur in the derived clause (c± V C2). Hence, for k = j — 1 down to i + 1 we successively 
conclude that 

Pr(Q k+ ix k+1 . . . Q n x n : <p[t{x\)/x\\ . . . [r(:cfc_i)/x fc _i][true/x fc ]) = p, 
Pr(Q k+ ix k+1 . . . Q n x n ■ <p[t(xi)/xi] . . . [r(a:fc_i)/xfc_i][false/a;jfc]) = p. 

From case k = i + 1 the lemma follows. □ 

Corollary 3.2 (Soundness of S-resolution). If the empty clause P is derivable by S- 
resolution from a given SSAT formula Q : 92 then Pr(Q : <p) = p. □ 

Corollary 13.21 follows directly from Lemma 13.11 namely for the special case dP = P . 
Theorem 13.31 shows completeness of S-resolution. 

Theorem 3.3 (Completeness of S-resolution). If Pr{Q :</?)= p for some SSAT formula 
Q : (p then the empty clause P is derivable from Q: ip by S-resolution. 

Proof. If G (p, i.e. tp contains the empty clause, then p = and the empty clause 0° is 
derivable by rule lR.1 1 In the remaining proof, we assume that ^ (p. We prove the theorem 
by induction over the number of quantifiers in the quantifier prefix Q. For the base case 
Q = Qx we distinguish three cases: 1) tp = (-<x) A (x). Then p = 0, and (^x) , (x)° 
are derivable by IR.1 [ and IR.3I finally yields 0°. 2) <p> = (-<x). Clauses (-ix)° and (x) 1 are 
derivable by IR.1 l and IR.2[ respectively, the latter since |= ^[false/a;]. If Q = 3 or Q = H Px 
then p = 1 or p = (1 — p x ), and 1 or 0( 1_ P a; ) can be derived bv lR.3l respectively. 3) <p = (x). 
Analogously to 2), if Q = 3 or Q = U Px then p = 1 or p = p x , and 1 or $ Px can be derived 
bv lR.3l respectively. 

In the induction step, we show that P is derivable for Pr{Qx Q : tp) = p. Let 
pi = Pr(Q : (p[tT\xe/x]) and P2 = Pr(Q : ip[± alse/x]). Induction hypothesis assumes 
that Pl and P2 are derivable from Q : ip[true/x] and Q : <^[f alse/x]. Applying the res- 
olution sequence deriving P1 from Q : </?[true/x] on Qx Q : <p yields either P1 or (-ix) pi . 
Analogously, either P2 or (x) P2 is derivable from Qx Q : <p. If Pl (respectively, P2 ) was 
derived then p = p\ (respectively, p = P2) by Corollary 13.21 (Note that if both P1 and P2 
are derivable then p\ = P2.) Otherwise, i.e. (-ix) pi and (x) P2 are derived, application of IR.3I 
gives p . □ 

The above presentation of S-resolution differs slightly from |TF10| in order to avoid 
overhead in interpolant generation incurred when employing the original definition, like the 
necessity of enforcing particular resolution sequences. For readers familiar with |TF10j . the 
particular modifications are: 1) derived clauses c p may also carry value p = 1, 2) former 
rules R.2 and R.5 are joined into the new rule IR.21 and 3) former rules R.3 and R.4 are 
collapsed into rule IR.31 These modifications do not affect soundness and completeness of 
S-resolution, confer Corollary 13.21 and Theorem 13.31 The advantage of the modification is 
that derivable clauses c p are forced to have a tight bound p in the sense that under each 
assignment which falsifies c, the satisfaction probability of the remaining subproblem exactly 
is p, confer Lemma 13. II This fact confirms the conjecture from [TF10I page 14] about the 
existence of such clauses (c V l) p and allows for a generalized clause learning scheme to be 
integrated into DPLL-SSAT solvers: the idea is that under a partial assignment falsifying 
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c, one may directly propagate literal I as the satisfaction probability of the other branch, 
for which the negation of i holds, is known to be p already. 

Example of S-resolution. Consider the SSAT formula <3? = U°' s xi 3x 2 ^°' 3 %3 ■ ((^1 V X2) A 
(-12E2) A {x 2 V 2:3) ) with Pr($) = 0.24. Clauses {x\ V x 2 )°, (^x 2 )°, {x 2 V x 3 )° are then 
derivable by IR.1 [ As x\ = true,X2 = false, X3 = true is a satisfying assignment, H fR^l 
(-.xi V x 2 V Then, ((-.xi V x 2 V -.a*) 1 , (x 2 V x 3 )°) hU (^1 V x 2 ) a3 , ((-x 2 )°, (-.xi V 

x 2 ) - 3 ) hisK-xi) - 3 , (N«(xivx 2 f) hEi(^) ' and finall y ((^i)°- 3 ,(xi)°) hEi® - 24 - 



3.2. Interpolating resolution for SSAT. We now devote our attention to the com- 
putation of generalized Craig interpolants for SSAT by means of an enhanced version of 
S-resolution, which is akin to resolution-based Craig interpolation for propositional SAT 
by Pudlak |Pud97] , We remark that on SSAT formulae Q : (A A B), Pudlak's algorithm, 
which has unsatisfiability of A A B as precondition, will not work in general. When instead 
considering the unsatisfiable formula A A B A —>Sa,b with —>Sa,b hi CNF then Pudlak's 
method would be applicable and would actually produce a generalized Craig interpolant. 
The main drawback of this approach however is the explicit construction of ^Sa.b, calling 
for explicit quantifier elimination. 

In the following, we propose an algorithm based on S-resolution for computing general- 
ized Craig interpolants which operates directly on A A B without adding and thus 
does not comprise any preprocessing involving quantifier elimination. For this purpose, the 
rules of S-resolution are enhanced to deal with pairs (c p ,I) of annotated clauses c p and 
propositional formulae /. Such formulae I are in a certain sense intermediate generalized 
interpolants, i.e. generalized interpolants for subformulae arising from instantiating some 
variables by partial assignments that falsify c, confer Lemma 13.41 Once a pair (0 P ,I) com- 
prising the empty clause is derived, / thus is a generalized Craig interpolant for the given 
SSAT formula. This augmented S-resolution, which we call interpolating S-resolution, is 
defined by rules IRI.1l IRI.2I and IRI.3I The construction of intermediate interpolants I in 
IRI.1 1 and IRI.3I coincides with the classical rules by Pudlak [Pud97| , while IRI.2I misses a 
corresponding counterpart. The rationale is that IRI.21 (or rather [RT2|) refers to satisfying 
valuations r of A A B, which do not exist in classical interpolation. As A A B becomes a 
tautology after substituting the partial assignment r from IR.2I into it, its quantified vari- 
ant Sa,b = 3ai, . . . , 61, . . . : A A B also becomes tautological under the same substitution 
Sa,b[t(xi)/xi, . . . , T{xi)/xj\. Consequently, ~>Sa,b[t(xi)/xi, . . . , r(xj)/xj] is unsatisfiable, 
and so are (A A -iSa,b)[t(x{)/xi, . . . ,r(xj)/xi] and { B A - <Sa,b)[t(xi)/xi, . . . ,T(xj)/xj]. 
This implies that the actual intermediate interpolant in IRI.21 can be chosen arbitrarily over 
variables in Vab- This freedom will allow us to control the geometric extent of general- 
ized interpolants within the "don't care" -region provided by the models of Sab, confer 
Corollary 13.61 

false ; c € A 



true ;c^B 



(cP,I) 

R2\c p ,I is any formula over Vab 



(RI.2) 
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((ci V 1 , (pa V ( Cl V c 2 )f, 

/i v l 2 ; x e Vk 
hAl 2 ;xeV B 
(-.x v Ji) a (x v J 2 ) 

77 r rr (KI.OJ 

((ciVc 2 )p,/) V 7 

The following lemma establishes the theoretical foundation of computing generalized Craig 
interpolants by interpreting the derived pairs (c p ,I). 

Lemma 3.4. Let $ = Q : (AAB) with Q = Q\x± . . . Q n x n be some SSAT formula, and the 
pair (c p ,I) be derivable from $ by interpolating S-resolution, where Q(c) = Q\X\ . . . Q%Xi. 
Then, for each r : Var{A A B) 4-i— > B with Vx € Var(c) : t(x) = ff c (x) it holds that 

(1) Var(I)QV AB , 

(2) Pr(Q i+1 x i+1 . . . Q n x n : (A A ~^Sa,b A ->I)[t(x{)/xi] . . . [r(xi)/xi]) = ; and 

(3) Pr(Q i+1 x i+1 . . . Q n x n : (I A B A -'5_a )B )[t(xi)/xi] . . . [r(xi)/xj]) = 0. 

Proof. We prove the lemma by induction over application of the interpolating S-resolution 
rules EEH EES and lROl In the base case, we can just apply IRTT1 and lRL2l Item CD clearly 
holds for both rules since I contains only variables in Va,b- Let us consider IRI.1 1 first. If 
c G A then / = false. By construction of r, i.e. c evaluates to false under r, it follows 
that A[t(x\) / x\\ . . . [r(xj)/xi] is unsatisfiable and thus 

Pr(Q' : (A A ^S a ,b A -7)[r(xi)/xi] . . . [r(xi)/xi]) = . 

As I = false, immediately 

Pr(Q' : (I A B A -«S A b)[t(xi)/xi] . . . [rfc)/^]) = . 
If c G -B then / = true. Obviously, 

Pr(Q' : (A A A -I)[r(xi)/xi] . . . [t^/h]) = 

and by construction of r, 

Pr(Q' : (I A B A ^[t^/h] . . . [rfc)/**]) = . 

For rule lRI.2| we have \={AA S)[r(xi)/xi] . . . [r(xi)/xj] which immediately implies that |= 
(3ai, . . . ,a a ,6i, . . . ,bp : (AAB))[r(xi)/xi] . . . [r(xi)/xj], i.e. |= <Sa,b[t(xi)/xi] . . . [r(xj)/xj] 
by definition of Sa,b- Rephrasing the latter, -^Sa,b[t{x{) / x\] . . . [r(xi)/xi] is unsatisfiable. 
Consequently, for any propositional formula I 

Pr(Q' : (A A ^S A ,B A -7)[r(xi)/xi] . . . [r(xi)/xi}) = , 

Pr{Q' : (I A B A -<S A b)[t(xi)/xi] . . . [rfcO/a*]) = . 

This proves items [2] and [3] for the base case. 

In the induction step, we now assume that the lemma holds for all clauses in the premises 
of rule IRI.3I Then, by construction of I, item Q] clearly holds for /, i.e. Var(I) C Va,b- 
Induction hypothesis assumes that 

Pr(Q' : (A A ^S A ,B A -Ii)[ti(xi)/xi] . . . [nfo-i)/si-i][true/a;j]) = , 

Pr{Q' :{hABA -5 A s)[r 1 (x 1 )/z 1 ] . . . [ti (xj-iVx^Htrue/x^]) = 
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holds for ((ci V ^Xj) Pl ,h) and for each n : Var(A A 5) B with Vx G Var(ci) : 

n{x) =ff Cl (x), and that 

Pr{Q! : (A A -«S A ,B A -/ 2 )[r 2 (xi)M] . . . [T 2 (xj_i)/x,,_i][f alse/x,,]) = , 

Pr(Q' :(/ 2 ABA -«Sa,b)[t 2 (xi)/xi] . . . \T 2 {x j - 1 )/x j - 1 }[fa.\se/x j \) = 

holds for ((c 2 VXj) P2 ,/ 2 ) and for each r 2 : Var(AAB) B with Vx G Var(c 2 ) : r 2 (x) = 

ff C2 (x), where j > i + 1 and Q' = Qj+iXj+i . . . Q n x n - Let r : Var(A A -B) |j-i— >• B be any 
assignment with r(x) = T\(x) if x G Var(ci) and r(x) = r 2 (x) if x G Var(c 2 ). Note that r 
is well-defined as ^ (ci V c 2 ), i.e. for each x G Var(c\) PI Var(c 2 ) : Ti(x) = r 2 (x). We now 
show that 

Pr A := Pr(Q jXj Q' : (A A -«S A ,B A -/)[r(xi)/xi] . . . [r(xj_i)/xj_i]) = , 

Pr B := Pr(Q jXj Q' : (I A B A -5 a ,b)[t(x 1 )/x 1 ] . . . [t(xj_i)/xj_i]) = 

by proving that 

Pr A , x ■■= Pr{Q! : (A A ^S a ,b A -I)[t(xi)/xi] . . . [t(xj_i)/Sj_i] [true/xj]) = , 
Pr A ,^ := Pr(Q' : (A A -«S A ,B A -7)[r(xi)/xi] . . . [r(^-_i)/xj_i][f alse/xj]) = , 

Pr B , x := Pr(Q' : (/ A B A -5 A s)[r(x 1 )/x 1 ] . . . [t(xj_i)/xj_i] [true/a:,]) = , 

Pr B ^ x := Pr(Q' : (/ A B A -5 A s)[r(x 1 )/x 1 ] . . . [r(xj_i)/a;j_i][f alae/xj]) = . 

We therefore distinguish the three cases Xj G V A , Xj G Vb, and Xj G V A) b- 

First, let be Xj G V A . Then, I = I± V 7 2 . By induction hypothesis and by construction 

of /, 

= Pr{Q! : (A A -<S A B A -/i)[ti(xi)/xi] . . . [ri(xj_i)/xj_i][true/a;j]) 

> Pr(Q' : (A A -.5a,b A -A A -I 2 )[ti(xi)/xi] . . . [ri(xj_i)/xj_i][true/xj]) 
= Pr(Q' : (A A -.5a,b A -J)[ri(xi)/xi] . . . [n(a:j-i)/xj_i][true/xj]) . 

Due to construction of r, it holds in particular that 

= Pr Ax . 
Analogously, 

= Pr(Q' : (A A -,S a ,b A -/ 2 )[r 2 (x 1 )/x 1 ] . . . [ts(xj_i)/xj_i] [false/a;,]) 

> Pr(Q' : (A A -.5a,b A -Ji A -7 2 )[r 2 (xi)/xi] . . . [^(xj_i)/xj_i][f alse/x^]) 
= Pr(Q' : (A A -<S A b A -/)[r 2 (xi)/xi] . . . [75(xj_i)/xj_i][f alse/x^]) 

and thus 

= Pr A ^ x . 

As Xj ^ Var(I) U Var(S) U Var^S^s), for each v G B it holds that 

Pr(Q' : (I A B A -5 A , B )[r(x 1 )/x 1 ] . . . Kx^Vx^^/x,]) 
= Pr{Q' :(/ABA -5 A , B )[r(x 1 )/x 1 ] . . . [t(xj_i)/xj_i]) 
which implies Prs, x = Ptb^x- We conclude from induction hypothesis that 
Pr(Q' : (h A B A -<Sa,b)[ti(xi)/xi] . . . [7i(zj-i)/ay_i]) = , 
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Pr(Q' :(/ 2 AfiA ^S A ,b)[t 2 (xx)/xx] . . . [^[xj-^/xj-x]) = 

again by virtue of Xj £ Var(I) U Var(B) U Var(->SA,B)- Moreover, 

Pr{Q! : (Jx A B A ^b)^^)/^] . . . [r^-O/xj-i]) = , 

Pr(Q' :(I 2 ABA -<Sa,b)[t(xi)/xi] . . . [r^-iV^-i]) = 

due to construction of r. Note that if Pr(Q : 991) = and Pr(Q : ip 2 ) = then Pr(Q : 
((pi V 972)) = since Pr(Q : ip) = if and only if ip is unsatisfiablell As a consequence, 



= Pr [Q 1 



(Ii A P A ^S A! b)[t(xx)/xx] ■ ■ ■ [T(xj-x)/xj-x] 
V (I 2 ABA -S AB )[r(xx)/xx] ■ ■ ■ [rixj^/xj-x] 

= Pr{Q! : ((Ji VJ 2 )ABA -nSA,B)[r(xi)/xi] . . . frfo-i)/^--!]) 

= Pr(Q' :(/ASA -<Sa,b)[t(xi)/xi] . . . [r^O/x^i]) 

= frs^ = Pr B ,^x ■ 

Second, let be Xj G Vb- Then, I = h A I 2 . As z 3 - ^ Var(A) U 7ar(-.5^ jB ) U Vbr(^I), 
with the same argument as above, 

n_ P r fo'. ( (AA^S AtB A^I 1 )[T(xx)/xx]...[T(x j -x)/x j -x\ \\ 
" ^ r ^ • V v ( A A A -J 2 )[r(x 1 )/x 1 ] . . . [rixj-ij/xj-!] J J 

= Pr(Q' : (A A A (-Ji V -n/ 2 ))[r(xi)/xi] • • • [rix^/xj-x]) 

= Pr(Q' : (A A -i<Sa,b A -./)[r(xi)/xi] . . . [r^-O/xj-i]) 
= P^A.z = P^A,-x • 
Again following the reasoning above, we have 

= Pr(Q' :(IxABA ^S a ,b)[tx{xx)/xx] . . . [nfo-i)/^-!] [true/xj]) 

> Pr(C : (/1 A h A B A ^Sa,b)[ti(xi)/xi] . . . [Ti(x i _i)/s i _i][true/x i ]) 
= Pr{Q! : (I A B A -<S A)B )[ri(xi)/xi] . . . [nixj^/xj-xlitme/xj]) 

and thus 

= Pr B>x 
as well as 

= Pr(Q' :(I 2 ABA -^ jB )[r 2 (xi)/xi] . . . fofo_i)/a^_i][f alse/zj]) 

> Pr(C : (/1 A / 2 A 5 A -5 A ,B)[r 2 (x 1 )/x 1 ] . . . [ 7 5(x i _i)/s i _i][f alse/xj]) 
= Pr(Q' : (I A B A -5 a ,b)[t 2 (x 1 )/x 1 ] . . . ^{x^/xj-xW alse/x^]) , 

and thus 

= Pr Bj ^ . 

Third, let be Xj € Va,b- Then, / = (->Xj V Ii) A (x^ V J 2 ), and we deduce 
= Pr(Q' : (A A ^S a ,b A -Ii)[ri(xi)/xi] . . . [rifo-iV^-iHtrue/xj]) 



^This statement is not true in general if Q also contains universal quantifiers, which is not the case in 
this article. However, extensions of SSAT involving universal quantifiers have also been considered in the 
literature, confer |Maj09| . 
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= Pr(Q': (AA^S A ,B 

A (( Xj A -Ji) V (^ Xj A -I 2 )))[Ti(xi)/xi] . . . [nfa-d/xj-^tTue/xj]) 
= Pr{Q' : (A A ^S A ,B A -/)[ti(xi)/xi] . . . [Ty{x j - 1 )/x j - 1 ] [true/xj]) 
and, in particular, 

= Pr AjX . 
Analogously, 

= Pr{Q! : (A A ^S a ,b A -/ 2 )[r 2 (xi)/xi] . . . [r 2 (^-_i)/^_i][f alse/xj]) 
= Pr(Q': (AA^S a ,b 

A ((a?j A -Ji) V (-.Xj A -J 2 )))faj(a;i)/si] • • • ^(xj-^/x^i]^ alse/xj]) 
= Pr(Q' : (4 A -<S A b A ^I)[t 2 {x 1 )/x 1 ] . . . [^(xj-J/xj-^ alse/xj}) 

and, in particular, 

= Pr A ^ x . 

Furthermore, 

= Pr(Q' :(hABA -5 A b)[ti(x 1 )/x 1 ] . . . [n^^O/x^^ftrue/^]) 
= Pr(Q' : ((-.Xj V Ji) A V I 2 ) A B 

A -><S J 4, ) B)[Ti(a;i)/a;i] . . . [ri(xj_i)/xj_i][true/a;,,-]) 
= Pr(Q' : (/ A B A -<Sa,b)[ti(xi)/xi] . . . [nfo-i)/si-i][true/a;j]) 

and, in particular, 

= Pr B , x ■ 
Finally, 

= Pr{Q' :{I 2 ABA -5 A b)[t 2 (x 1 )/x 1 ] . . . ^(xj^/x^f alse/xj]) 
= fV(C : ((-Xj V Ii) A (xj VJ 2 )AB 

A ^5 j4 ,b)[t 2 (xi)/xi] . . . [r 2 (xj_i)/xj_i][false/xj]) 
= Pr(Q' : (I A B A -<S A b)[t 2 (x 1 )/x 1 ] . . . [^(xj-O/x^i] [f alse/aij]) 

and, in particular, 

= Pr B ,-rc ■ 

Having shown that Pr Ax = P r A,^x = P r B,x = P r B,^x = 0, we can now prove the interme- 
diate result above, i.e. Pr A = Pr B = 0. If Qj = 3 then Pr A = T&ax.{Pr A ^ x , Pr Aj -, x ) = and 
Pr B = max(Pr BjX , Pr B ^ x ) = 0, and if Qj = U Px then Pr A = p x -Pr AyX +{l-p x )-Pr A ^ x = 
and Pr B = p x ■ Pr B , x + (1 - p x ) ■ Pr B ^ x = 0. 

To finish the proof, we finally need to show that items [2] and [31 i.e. 

Pr(Q i+1 x i+ i . . . Q n x n : (A A ^S AjB A -J)[r(xi)/xi] . . . [r(xi)/xi]) = , 
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Pr(Q i+ ix i+ i . . . Q n x n ■ (I A B A ->Sa,b)[t(xi)/xi] . . . [r(xi)/xi]) = , 

follow from Pva = Ptb = 0, i.e. from 

Pr(QjXj . . . Q n x n : (A A ->Sa,b A -~I)[t(x 1 )/xi] . . . [t(xj_i)/xj_i]) = , 

Pr(QjXj . . . Q n x n : (/ABA ^S a ,b)[t{xi) /x{\ . . . [t(x.,-_i)/xj_i]) = . 

If j = i+1 then the result is obvious. Otherwise, i.e. if j > i + 1, the variables x,+i, . . . ,Xj—x 
do not occur in the derived clause {c\ V c 2 ) since Q{c\ V C2) = Q\X\ . . . QiX{. By definition 
of assignment r, for k = j — 1 down to i + 1 we may therefore successively conclude that 

Pr{Qk+\Xk+i ■ ■ ■ Q n x n ■ 

(A A -&a,b A ->I)[t(x 1 )/x 1 ] . . . [r(xk-i)/x k -i][tTae/xk\) = , 

• • QnXn ■ 

(A A ->Sa,b A -J)[t(xi)/xi] . . . [r(x fc _i)/xfc_i][false/x fc ]) = , 

• • QnXn ■ 



Pr(Q k +ix k+ i 
Pr(Q k+1 x k +i 
Pr(Q k+1 x k+ i 



(I A B A ->Sa,b)[t(xi)/xi] . . . [r(xfc_i)/xfc-i][true/x fc ]) = , 

• • QnXn ■ 

(I A B A ^5a,b)[t(xi)/xi] . . . [r(x fc _i)/x fc _i][f alse/x fc ]) = . 

From case k = i + 1 the result immediately follows. □ 

Completeness of S-resolution, as stated in Theorem l3.3l together with above Lemma [3.4l 
applied to the derived pair (0 P , I), yields 

Corollary 3.5 (Generalized Craig interpolants computation). If Q : {A A B) is an SSAT 
formula then a generalized Craig interpolant for (A, B) can be computed by interpolating 
S-resolution. □ 

Note that computation of generalized interpolants does not depend on the actual truth 
state of A A B. The next observation facilitates to effectively control the geometric extent 
of generalized Craig interpolants within the "don't care" -region Sab- This result will be 
useful within applications of generalized Craig interpolation to the symbolic analysis of 
probabilistic systems being investigated in Section [H 

Corollary 3.6 (Controlling generalized Craig interpolants computation). If I = true is 
used within each application of ro/e fRI.21 then Pr(Q : (AA—iZ)) = 0. Likewise, if I = false 
is used in rule EE21 then Pr(Q : (Z A B)) = 0. 

Proof. The proof works analogously to the one of Lemma 13.41 For the base case, it is 
clear that the desired property for IRI.1I is independent of ->Sa,b- For IRI.2I if I = true 
then clearly Pr(Q' : (A A ->I)[t(xi)/xi] . . . [r(xi)/xi]) = 0, and if I = false then Pr(Q' : 
(/ A B)[t{x\) / x\] . . . [r(xj)/xj]) = 0. Then, we can modify the induction hypothesis: for 
case ll I = true in IRI.21 ' . we assume that Pr{Q! : (A A -^I\)\t\(x\) / x-±\ . . . [true/xj]) = 0, 
Pr{Q' : (A A -/ 2 )[t 2 (xi)/xi] . . . [false /xj]) = 0, and for "I = false in EES]' that Pr(Q' : 
(Ji A B)[t x {x x )/xi] . . . [true/xj]) = 0, Pr(Q' : (J 2 A S)[r 2 (xi)/xi] . . . [f alae/xj]) = 0. The 
induction step then follows the same reasoning as in the remaining proof of Lemma I3.41 □ 
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Figure 3: Example of interpolating S-resolution and illustration of the resulting generalized 
Craig interpolants by means of Karnaugh- Veitch diagrams. Arrows denote appli- 
cations of the specified interpolating S-resolution rules, while DC stands for any 
formula over Va,b as in rule IRI.21 



Observe that the special interpolants Z from Corollary 13. 6l relate to the classical strongest 
and weakest Craig interpolants A 3 and B , respectively, in the following sense: Pr(Q : 
(AA-iI)) = Oiff^A^Xiff^ Vai, ...,a a :(A 1) iff |= (A 3 1), as a h . . . , a a do not 
occur in 1. Analogously, Pr(Q : (1 A B)) = iff (= X => ^B iff |= V&i, . . . , bp : (1 -..B) 
iff |= J =>. B V . 
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Figure 4: A simple MDP M. 



Example of computing generalized Craig interpolants by interpolating S-resolution. For an 
example of interpolating S-resolution, consider the SSAT formula $ = H ' a 3x 
(AAB) with A = ((y)A(aV-uc)) and B = ((x) A (->y V6)). Then, V A = {a}, V B = {&}, and 
Va,b = {x, y}- It is not hard to see that the only satisfying assignment r of the propositional 
formula A AB is given by r(a) = true, r(x) = true, r(y) = true, and r(6) = true. Hence, 
Pr($) = 0.12. A derivation of the empty clause 012 together with its associated generalized 
Craig interpolant -ucV (y ADC) is shown in Figure El while DC stands for any formula over 
variables in Va,b as in rule IRI.21 Note that pair (( _, a V —»x V —>y V -<b) 1 ,DC) is derivable 
by rule fRL2l since \= (A A B)[t{o) / o\[t{x) / x\[T{y)/y][T(b) /b]. Applying Corollary Ej3 by 
choosing DC = true and DC = false, we obtain the generalized Craig interpolants X\ = 
-<x\/y andX 2 = ->x, respectively, such that Pr(Q : (AA^li)) = and Pr(Q : (X 2 AB)) = 0. 
In other words, A => X\ and X 2 =>■ ~>B, as illustrated by the Karnaugh- Veitch diagrams in 
Figure El 

4. Applications of generalized Craig interpolation to analysis of 

probabilistic systems 

In this section, we investigate the application of generalized Craig interpolation to the 
symbolic analysis of probabilistic systems. We direct our attention to two analysis goals, 
namely to probabilistic state reachability in Section 14.11 as well as to probabilistic region 
stability in Section 14.21 As a system model, we consider finite-state Markov decision pro- 
cesses (MDPs) [Bel57 . An MDP A4 = (i, S, Act,ps(-, •, •)) is a finite-state system in which 
state changes are subject to non- deterministic selection among available actions followed 
by a probabilistic choice among potential successor states, while the probability distribu- 
tion of the latter choice depends on the selected action. More precisely, S is a finite set 
of states, i € S is the initial state, Act is a finite set of actions, and ps(s,a,s') gives the 
probability that A4 performs a transition step from s € S to s' € S under action a € Act. 
For an example, consider the simple MDP Ai from Figure [4] where i = i, S = {i,f,e,s}, 
and Act = {a, b}. A transition ps(z, act,z') = p > is indicated by an arrow from z to 
z' accompanied by action act and by the corresponding transition probability p. If two 
states are not connected by an arrow then the corresponding transition probability is 0, 
and if no action is specified then that transition is feasible for all actions. A probability 
measure of an MDP is well-defined only if considering a particular scheduler a resolving 
the non-determinism. That is, a schedules the action for the current state. Different such 
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schedulers a have been investigated in the literature, confer, for instance, [BHKH05]: a 
may select the next action either in a deterministic or randomized fashion. In both cases, 
a may have access to and thus base its selection on either the current state only or the full 
system history. In our scenarios, we do not manipulate schedulers explicitly, but define the 
probability measures obtained by worst-case deterministic schedulers achieving maximum 
or minimum, depending on how the worst case is understood, probability of reaching target 
states directly as the limit of a recursive function over N. For each k € N, the recursive 
function determines the maximum or minimum probability of reaching target states within 
k steps, as achieved by a worst-case history-dependent scheduler. As a worst-case history- 
dependent scheduler will always maximize or minimize the probability of reaching the target 
within the remaining number of steps, its performance coincides with the probabilities com- 
puted by a backward induction resolving non-deterministic choices by taking the maximum 
or minimum, respectively, of the probability values obtained from the next-lower recursion 
depth. 

All experiments mentioned in this section were performed on a 1.83 GHz Intel Core 2 
Duo machine with 1 GByte physical memory running Linux. 



4.1. Interpolation-based probabilistic state reachability. Let be given an MDP M. 
and a set of target states Target C S in Ad. With regard to probabilistic state reachability, 
the goal is to compute the probability of reaching the target states Target from the initial 
state i under some explicitly or implicitly (e.g., by an optimality condition) given scheduler 
a. In most applications, the target states are considered to be bad, for instance, to be fatal 
system errors, such that one is faced with computing the worst-case probability of reaching 
the bad states, i.e. maximizing the reachability probability under each possible scheduler. 
This maximum probability MaxReach(A4, Target) can be defined directly as the limit of 
the maximum step-bounded probability of reaching the target states as similarly shown 
by [FHH+lll Lemma 1], i.e. 

MaxReach (A4, Target) = \im MaxReach k M Target {i) 



where 



MaxReach k MTarget {s) 



1 ; s £ Target 

;s i Target, k = 

max Yl ps(s,a,s') ■ MaxReach^^ As') ;s ^ Target, k > 

a&Act s , £S ' » 



gives the maximum probability of reaching the target states from state s G S within k 
steps (k £ N) under each possible scheduler. For some threshold value 6 £ [0, 1], the safety 
verification problem is to decide whether the worst-case probability of reaching the bad 
states is at most 9, i.e. to decide whether 

MaxReach(M, Target) < (4.1) 

holds. 

In previous work [FHT08, FTElOl |TEF11| . we have established a symbolic falsification 
procedure for above problem l4.11 Though this approach is based on SSMT, i.e. an arithmetic 
extension of SSAT, and works for the more general class of discrete-time probabilistic hybrid 
systems, which roughly are MDPs with arithmetic-logical transition guards and actions, 
the same procedure restricted to SSAT is applicable for finite-state MDPs. The key idea 
here is to adapt bounded model checking (BMC) [BCCZ 99J to the probabilistic case by 
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encoding step-bounded reachability as an SSAT problem: like in classical BMC, the initial 
states, the transition relation, and the target states of an MDP A4 are symbolically encoded 
by propositional formulae in CNF, namely by Init(s), Trans (s,nt,pt, s'), and Target(s), 
respectively, where the propositional variable vector s represents the system state before and 
s' after a transition step. To keep track of the non- deterministic and probabilistic selections 
of transitions in Trans(s, nt,pt, s'), we further introduce propositional variables nt and pt 
to encode non-deterministic selection among available actions and to describe probabilistic 
choice of the successor state, respectively. Assignments to these variables determine which 
of possibly multiple available transitions departing from s is taken. In contrast to traditional 
BMC, all variables are quantified: all state variables s and s' are existentially quantified in 
the prefixes Q s and Q s /. The transition-selection variables nt encoding non-deterministic 
choice are existentially quantified by Q,nt± while the probabilistic selector variables pt are 
bound by randomized quantifiers in Q p t|j For the sake of clarity, let be t := nt U pt and 

Qt '■= QntQpt- 

According to [FHT08, Proposition 1], the maximum probability of reaching the target 
states in A4 from the initial state % within k transition steps, i.e. MaxReachj^ Tar g et 
equal to the satisfaction probability 

states reachable within k steps n 't target states 

lb k ■= Pr(Q{k) : (lnit(s ) A /\ k . =i Trans{si-\, tj, Sj) A f\/^ = o Tar 9 et ( s ^ )) ( 4 - 2 ) 

with Q(k) := Q so Q tl Q S1 ... Q Sk _, Q tk Q Sk . 

Observe that each value lbk = MaxReachj^ Targeti 1 ) can be computed by an SSAT solver 
and constitutes a lower bound of the maximum reachability probability MaxReach(A4, Target) 
due to monotonicity of the chain (MaxReachj^^Target i 1 )) k^n' ^ ms symbolic approach, called 
probabilistic bounded model checking (PBMC), is able to falsify safety properties of shape I4TT1 
once a value lbk > & is computed for some k. 

However, the development of a corresponding counterpart based on SSAT that is able to 
compute upper bounds ubk of the maximum reachability probability MaxReach(A4, Target) 
was left as an open challenge. Such an approach would permit to verify safety properties 
of shape EL"T1 once a value ubk < is computed for some k. 

In the remainder of this section, we propose such a symbolic verification procedure for 
above problem 14.11 by means of generalized Craig interpolation. This verification method 
proceeds in two phases. Phase 1 computes a symbolic representation of an overapproxima- 
tion of the backward reachable state set, where a state is backward reachable if it is the origin 
of a transition sequence leading into Target. Phase 1 can be integrated into PBMC, as used 
to falsify the probabilistic safety property. Whenever such falsification fails for a given step 
depth k, we apply generalized Craig interpolation to the (just failed) PBMC proof to com- 
pute a symbolic over approximation of the backward reachable state set at depth k and then 
proceed to PBMC at some higher depth k' > k. As an alternative to the integration into 
PBMC, interpolants describing the backward reachable state sets can be successively ex- 
tended by "stepping" them by prepending another transition, as explained below. In either 
case, phase 1 ends when the backward reachable state set becomes stable, in which case we 

^Non-deterministic branching of n alternatives can be represented by a binary tree of depth [log 2 n] and 
probabilistic branching by a sequence of at most n — 1 binary branches, yielding [log 2 n\ existential and 
n — 1 randomized quantifiers, respectively. 
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have computed a symbolic overapproximation of the whole backward reachable state set. 
In phase 2, we construct an SSAT formula with parameter k that forces the system to stay 
within the backward reachable state set for k steps. The maximum satisfaction probability 
of that SSAT formula then gives an upper bound on the maximum probability of reaching 
the target states. The rationale is that system runs leaving the backward reachable state 
set will never reach the target states. 

Phase 1. Given an SSAT encoding of an MDP M. as above, the state-set predicate B k {s) 
for k € N over state variables s is inductively defined as 

• B°{s) := Target(s), and 

• B k+1 (s) :=B k (s)Vl k+1 (s) 

where X fc+1 (sj_i) is a generalized Craig interpolant for 

, , 

Trans(sj-i,tj, Sj) A B (sj), Init(so) A /\. ^ Trans (s^-i, ij, s«) J 

with j > 1 with respect to SSAT formula 

j - 1 steps "forward" (=B) one step « backward » ( =A) 

(7—1 / s \ 

Init(s ) A /\ Trans{s i - 1 ,t i ,s i ) A Trans(s j ^ 1 ,t j , Sj) AB k (sj) \ . (4.3) 

Observe that each generalized Craig interpolant I k+l (s) can be computed by interpolating 
S-resolution if we rewrite B k (s) into CNF, the latter being always possible in linear time by 
adding auxiliary T/4-variables. During computation of each I k+1 (s), we take I = true in 
every application of rule IRI .21 such that B k (s) overapproximates all system states backward 
reachable from target states within k steps due to Corollary 13.61 Whenever B k (s) has 
stabilized, i.e. 

B k+1 {s) => B k {s) , 

we can be sure that B(s) := B k {s) overapproximates all backward reachable states. It is 
obvious that B k (s) finally stabilizes in the finite-state case. 

Note that parameter j > 1 can be chosen arbitrarily, i.e. the system may execute any 
number of transitions until state is reached since this does not destroy the "backward- 
overapproximating" property of B k+1 (s). The rationale of having parameter j is the addi- 
tional freedom in constructing generalized interpolants since j may influence the shape of 
I k+1 (s), as we will see in the example below. 

We remark that phase 1 is a clean generalization of McMillan's approach [McM03, 
IMcM05| . the latter having unsatisfiability of A A B as precondition in each iteration k^ 



^Instead of overapproximating the backward reachable state set, McMillan's scheme )McM03l IMcM05| 
actually targets at forward reachable states, which however makes no fundamental difference in the non- 
probabilistic setting. 
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Phase 2. Having symbolically described all backward reachable states by the predicate B(s), 
upper bounds ub^ of the maximum probability MaxReach(A4, Target) of reaching the target 
states Target can now be computed by SSAT solving applied to 

states reachable within k steps stay in back-reach set 

ub k := Pr[Q{k) : (imt(so) A f\ i=i Trans(s^ 1 ,t i , Si ) A A- = o^ (Si) )) ' (4 ' 4) 

First observe that the formula above excludes all system runs that leave the set of backward 
reachable states. This is sound since leaving B(s) means to never reach the Target(s) states. 
Second, the system behavior becomes more and more constrained for increasing k, i.e. the 
u&fc's are monotonically decreasing. With regard to solving problem l4.lt the safety property 
MaxReach(Ai, Target) < 9 is verified by the procedure above once an upper bound ub k < 6 
is computed for some k. 

Example. To illustrate the symbolic approach to probabilistic safety verification based on 
generalized Craig interpolation, consider the simple MDP A4 from Figure H] with s being 
the only target state. 

With regard to the symbolic encoding of M, we introduce four Boolean variables i, f, e, s 
to describe the state space. The literal i means that Ai is in state i while literal —*i expresses 
that A4 is not in i. The same holds analogously for the other states. Note that, in order 
to encode valid system states, we have to ensure that exactly one of the variables i, f,e,s 
is true in each time instant. The encoding of this constraint will be explained later on. 
The non-deterministic choice between actions a and b is encoded by a Boolean variable 
act while action a is represented by the positive literal act and action b by the negative 
literal ^act. For the three probabilistic choices in ftA, we introduce three Boolean variables 
pi for the choice from i, pea for the choice from e under action a, and peb for the choice 
from e under action b. Recall that all state variables as well as variables encoding non- 
deterministic selection are existentially quantified while variables describing probabilistic 
choices are bound by randomized quantifiers. We thus obtain the corresponding quantifier 
prefixes 

Q s = 3i3f3e3s, 
Q t = 3act U°- 9 pi U°- 6 pea U°- 5 peb , 
Q s > = 3i' 3f 3e' 3s' . 
The formulae in CNF representing the initial state and the target states are specified by 

Init(s) = (i) A (->f) A (-.e) A (-.s) and Target(s) = (s) , 

respectively. To obtain the transition relation predicate, we encode each single transition 
step. For instance, a step from state e to / under action a can be encoded by the implication 
(e A act A ^pea) => /', the latter being equivalent to the clause (->e V -^act V pea V /'). The 
conjunction of all these clauses then encodes the full system behavior symbolically. Since 
we represent each system state by an own Boolean variable, as mentioned above, we need 
to enforce that exactly one of the primed state variables, constituting the system state after 
the transition step, carries value true. This is simply achieved by the formula in CNF 
exactly _one(i', f, e', s') = (if V/'Ve'V s') A H' V -./') A (-.»' V V) A (-.»' V -V) A (-./' V 
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Table 1: Experimental results of applying the generalized interpolation scheme [4.31 on J\A 
from Figure H] for different values of parameter j. In addition to the formal pre- 
sentation of the predicates, the concrete state sets are given explicitly. 



->e') A V -ia') A (—>e' V -is'). The transition relation predicate in CNF then is 

Trans(s, t, s') = (->i V pi V /') A (->i V ~^pi V e') 

A (^eV ^actV peaV f) A (-.e V ^aci V ^pea V a') 

A (-ie V act V pe6 V a') A (-.e V aci V ^pe6 V «') 

A (-i/V/') A (nsVs') A exactly. one(i', f',e',s') . 

We are now interested in the maximum probability of reaching the target state s from 
the initial state i. Applying the PBMC scheme 14.21 we are only able to compute lower 
bounds Ibk of the maximum reachability probability, for instance, Ibo = lb\ = 0, lb<i = Z63 = 
0.54, lb A = lbs = 0.693, lb 20 = 0.817971, lb wo = 0.818181 8181818 1803208. The 
latter results were achieved by employing the SSMT solver SiSAT@ [ TEFllj that provides 
a convenient input language for specifying probabilistic transition systems like MDPs. Un- 
winding of the system's transition relation for increasing step bounds k, i.e. the construction 
of the SSAT formulae specified by scheme 14.21 in our context, is done fully automatically. 
Furthermore, several algorithmic optimizations are exploited to improve performance of 
the tool. Concerning runtime, all 100 SSAT formulae were solved within 37.05 seconds, 
while computation of the first 20 lower bounds Ibo to /&20 just needed 370 milliseconds. 
The highest computation time for a single SSAT problem was obtained for Z&ioo; namely 
1.14 seconds. The evolution of the Zfefe's up to k = 20 is presented graphically on the right 
of Figure [U Given these results, one can suppose that the lower bounds converge to and 
never exceed value 9 / n = 0.81. However, there is no mathematical guarantee for the latter 
guess. 

To overcome this limitation, we first apply the generalized interpolation scheme f4.3l to 
compute an overapproximation of the backward reachable state set. The latter then facil- 
itates to compute upper bounds ubk of the maximum reachability probability by means of 
scheme 14. 4 i In order to compute the generalized Craig interpolants I k+l (sj-±) automat- 
ically during solving the SSAT formulae 14.31 we have implemented a simple DPLL-based 
SSAT solver that integrates interpolating S-resolution. As mentioned earlier, scheme 14.31 
allows freedom in choosing parameter j > 1. This parameter permits to specify the number 
j — 1 of transition steps until system state is reached, which is the common state of 
formula parts A and B. The experimental results of applying the generalized interpolation 
scheme 14.31 on the MDP Ai for different values of j are shown in Table [TJ 



'The SiSAT tool is available onjhttpT77sisat.gforge.avacs.org/ 
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Figure 5: Illustration of the computed state sets for MDP M by the generalized interpola- 
tion scheme T4.3I with j = 3 (left), and lower bounds Ibk and upper bounds ubk of 
the maximum probability of reaching target state s over number k of transition 
steps computed by schemes I4.2I and f4.4| respectively (right). 

From the results of Table [H we observe that the value of j actually has an impact 
on the shape of the resulting interpolants. Let us consider the first interpolants X 1 which 
overapproximate all states backward reachable in one step. Clearly, the exact set of states 
backward reachable in one step is {e, s}. For j = 1, the overapproximated set {/, e, s} 
computed by the procedure is too coarse and actually contains a state which is not backward 
reachable at all, namely /. Though the set {i, e, s} for j = 2 actually consists of backward 
reachable states only, it is not tight enough as the initial state i is backward reachable 
after two steps only. For j = 3, we achieved the precise set {e,s}. Continuing the scheme 
for j = 1, X 2 and then Z 3 become true meaning that the overapproximated set of the 
backward reachable states B covers the whole state space. Using this inconclusive result in 
scheme 14.41 yields only trivial upper bounds ubk = 1 f° r all k. With regard to j = 2, the 
interpolation process has stabilized after computation of X 2 . The resulting state set {i, e, s} 
encoded by B actually is the precise set of all backward reachable states. Though X 1 was 
too coarse, this could be compensated in the computation of X 2 . For j = 3, we observe that 
all generalized interpolants X 1 , X 2 , and X 3 describe the corresponding backward reachable 
states accurately, thus leading to the precise set of all backward reachable states. The 
computed state sets for j = 3 are illustrated on the left of Figure [5j After having examined 
the results above, it seems that the greater the value of j, i.e. the more transition steps are 
performed, the more accurate the resulting overapproximation of the backward reachable 
state set. 

Concerning runtime, each generalized Craig interpolant was computed by the interpo- 
lating DPLL-based SSAT solver within fractions of a second, where the highest runtime of 
36 milliseconds was observed when computing Z 3 for j = 3. 

Having computed a symbolic representation B(s) of an overapproximation of all back- 
ward reachable states, we are now able to compute upper bounds ubk of the maximum 
reachability probability by means of scheme 14.41 where we use B(s) = —>f V s as obtained 
for j = 3 as well as for j = 2. Again employing the SSMT tool SiSAT, some of the results 
are ub = 1, ub x = ub 2 = 0.9, ub 3 = ub A = 0.855, ub 5 = ub 6 = 0.83475, . . ., ub 20 = 0.818243, 

ub wo = 0.81818181818181821948. Concerning runtime, all 100 SSAT formulae were 
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solved within 54.76 seconds, while computation of the first 20 upper bounds ubo to ub 2 o 
just needed 400 milliseconds. The highest computation time for a single SSAT problem 
was obtained for u&ioo, namely 1.77 seconds. The evolution of the ub^s up to k = 20 is 
presented graphically on the right of Figure [5j 

In addition to estimating the maximum reachability probability from below using the 
PBMC scheme 14.21 we are now able to estimate the probability also from above. In our 
example, we can safely conclude that 

0.81818181818181803208 = lb 100 < MaxReach(M, {s}) < ub l00 = 0.81818181818181821948 

holds where the difference u&ioo — ^ioo is below 10 -15 . The total computational effort for 
obtaining this precise result is about 92 seconds. If reduced accuracy suffices then runtime 
obviously improves. For instance, the fact 

0.817971 = lb 20 < MaxReach(M, {s}) < ub 20 = 0.818243 

with ub 2 o — ^20 < 10~ 3 was deduced within one second. With regard to the safety verifica- 
tion problem l4.H system safety for each threshold value 9 with 9 < 0.817971 or 9 > 0.818243 
is falsified or verified, respectively, within a second. 

With respect to competitive and more established methods based on value or policy 
iteration, we observed that the runtime of our prototypic tool chain does not compare favor- 
ably on the simple probabilistic reachability problem above. For instance, the version 4.0.1 
of the PRISM model checked [KNPllj solved the probl em in about 600 milliseconds with 
a precision of 10" 15 (returning the result 0.8181818181818175). 

In spite of the above fact, we have identified two promising directions for future research 
where probabilistic reachability analysis based on generalized Craig interpolation may pay 
off: 

(1) Embedding the same interpolation process into SSMT [FHT08], i.e. an arithmetic exten- 
sion of SSAT, renders the generalized Craig interpolation scheme [4731 directly applicable 
to probabilistic hybrid discrete-continuous systems, yielding a symbolic overapproxima- 
tion of the backward reachable state set. As for the finite state case, scheme [4.41 then 
facilitates computing upper bounds of the reachability probability for hybrid systems 
by means of SSMT solving, just as already pursued when computing lower bounds 
according to the PBMC scheme IP1 [FHT081 ITF081 IFTE101 ITEFll] . 

It is important to remark that classical value or policy iteration procedures are not 
directly applicable in the hybrid state case but even after finite-state abstraction, confer, 
for instance, [ZSR+lOl IfHH+11] . 

(2) Due to its symbolic nature, the analysis procedures based on SSAT and SSMT support 
compact representations of concurrent probabilistic (finite-state and hybrid) systems 
without an explicit construction of the product automaton [TEF11], the latter being of 
size exponential in the number of parallel components. This fact constitutes a strong 
argument that these symbolic procedures are able to alleviate the state explosion prob- 
lem, which arises necessarily when applying explicit-state algorithms or methods based 
on finite-state abstraction refinement. 



More information can be found on http://www.prismmodelchecker.org/ 
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4.2. Interpolation-based probabilistic region stability. In addition to probabilistic 
state reachability being investigated in the previous section, we now address the problem 
of probabilistic region stability. For that purpose, we take into account the notion of re- 
gion stability as introduced for non-probabilistic hybrid systems by Podelski and Wagner 
in |PW07al IPW07b] . According to their definition, given some set R of states called region, 
a (non-probabilistic) system is called stable with respect to region R iff for every infinite run 
(so, si, . . . , S{, . . .) of the system, i.e. for every infinite sequence of states that follows the 
transition relation, there is some point of time i > such that from i on the system remains 
in R forever, i.e. 3i > Vj > i : sj € R. 

Concerning the probabilistic case, several adaptations of region stability seem feasible, 
some of which pose measurability problems. Our main concern in this article being to iden- 
tify potential application areas for generalized Craig interpolation rather than to discuss 
semantic issues of probabilistic stabilization, we do study a simple notion of probabilistic 
region stability in the sequel which circumvents measure-theoretic issues. As for probabilis- 
tic state reachability, we aim at defining a reasonable probability measure as the limit of 
the value of a recursive function defining the corresponding step-bounded measures. In- 
tuitively, we consider finite run prefixes (so, s±, . . . , Sj) such that from time point i on the 
probabilistic system remains in the given region forever under each possible future behavior, 
i.e. independent of the non-deterministic and probabilistic choices the system will take. The 
latter fact is guaranteed whenever the system has reached an invariance kernel of the given 
region that can never be left. The probability measure is then defined by the minimum 
probability of reaching the maximal invariance kernel. 

Formally, let be given an MDP A4 and a set of states Region C S called the stabilization 
region or the region for short. An invariance kernel IC C Region with respect to Ai is a set 
of states from Region such that there is no transition from a state in /C to a state outside 
IC, i.e. there does not exist a tuple (z, act, z') £ fC X Act x (S \ IC) : ps(z, act, z') > 0. An 
invariance kernel IC is called maximal if adding any new states to IC does not lead to an 
invariance kernel, i.e. each IC U Z with Z C Region \ IC and Z ^ is not an invariance 
kernel. Note that the maximal invariance kernel is unique. The latter fact can be simply 
shown using the observation that the set of all invariance kernels IC C Region with respect 
to Ai is closed under union. Let IC* C Region be the (unique) maximal invariance kernel 
with respect to At. Then, the minimum probability MinStable(A4, Region) that Ai is stable 
with respect to Region is defined as the limit of the minimum step-bounded probability of 
reaching the maximal invariance kernel IC* , i.e. 



gives the minimum probability of reaching IC* from state s € S within k steps (k G N) 
under each possible scheduler. 

When considering stabilization within Region as the desired property then the value of 
MinStable(M, Region) establishes the probability of stabilizing in worst case, i.e. under an 
optimal adversarial scheduler. For some threshold value 8 € [0,1], the stability verification 
problem then is to decide whether this worst-case probability is at least 9, i.e. to decide 




where 
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whether 

MinStable(M, Region) > 9 (4.5) 

holds. 

In what follows, we propose a symbolic verification procedure for above problem 14.51 
In a first phase, we compute a symbolic representation of an invariance kernel by means of 
generalized Craig interpolation. The main idea here is to iteratively eliminate states z not 
belonging to an invariance kernel from Region until a fixed point is reached. Due to the use 
of interpolation, the set of such states z is overapproximated in each iteration, meaning that 
potentially too many states are removed. This implies that the resulting invariance kernel 
is not necessarily maximal. However, each invariance kernel can be used for computing 
valid lower bounds of MinStable(M. , Region) . The latter computation then is performed 
in a second phase by means of SSAT-based bounded reachability checking. Once a lower 
bound lb > 8 is computed, property 14.51 is verified. 

Phase 1. Let be given an SSAT encoding of an MDP Ai as explained in Section [4.11 as well 
as some propositional formula Region(s) encoding the stabilization region Region. Then, 
the state-set predicate lZ k (s) for k € N over state variables s is inductively defined as 

• 1Z°(s) := Region(s), and 

• K k+1 (s) := lZ k {s) A -iZ fc+1 (s) 

where X k+1 (sj-i) is a generalized Craig interpolant for 

=A =1 

A. / * V 

Trans(sj-i,tj,Sj) A^1Z k (sj), Init(so) A /\. ^ Trans(si-i, ti, Si) J 
with j >1 with respect to SSAT formula 

j - 1 steps "forward" (=B) Qne step « backward » from (=A) 

(A 3 ~ 1 ^ \ 

Init(so) A f\ _ Trans(s i - 1 , t i , Si) A Trans{s j - 1 ,t j ,Sj) A -Jl k (sj) J . (4.6) 

Observe that each T k+1 (s) can be computed by interpolating S-resolution if we rewrite 
—<7Z k (s) into CNF, the latter being always possible in linear time by adding auxiliary Va- 
variables. During computation of each I k+1 (s), we take / = true in every application 
of rule IRI.21 such that T k+1 (s) overapproximates all system states directly leading to the 
state set -<TZ k (s) due to Corollary 13.61 As a consequence, from each state in lZ k+1 (s) = 
TZ k (s) A -nl k+1 {s) it is infeasible to leave the set TZ k {s) in one step. Whenever the chain 
lZ k {s) has stabilized, i.e. 

K k (s) K k+1 {s) , 

it follows that K(s) := K k (s) is an invariance kernel of Region(s) with respect to M, i.e. 
once entered, the system cannot leave the set JC(s). Obviously, the chain lZ k (s) eventually 
stabilizes in the finite-state case. 

Similar to scheme 14.31 parameter j > 1 can be chosen arbitrarily, i.e. the system may 
execute any number of transitions until state 8j—\ is reached since this does not destroy the 
overapproximation property of I k+1 (s). The presence of parameter j gives us additional 
freedom in constructing generalized interpolants as j may influence the shape of Z k+1 (s), 
as we will see in the example below. 
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Phase 2. Having computed a symbolic representation tC(s) of a (not necessarily maximal) 
invariance kernel /C with respect to A4, we now compute lower bounds of the minimum 
probability MinStable(A4, Region) of stabilizing within Region by means of SSAT solving. 
To this end, first observe that MinReach k M ^» {%) is monotonic in k which implies that 
MinReach k M ^*(i) < MinStable(M, Region) for each k G N. Let K* be the unique maximal 
invariance kernel with respect to A4. Then, tC C K* since /C is an invariance kernel and the 
maximal invariance kernel JC* is unique. As a consequence, 

MinReach k MK {i) < MinReach k MK ,{i) 

for each k G N. Summing up, each value of MinReach k M ^(z) establishes a lower bound of 
MinStable(A4, Region). In principle, MinReach k M ^(i) can be reduced to an SSAT formula 
similar to PBMC scheme 14.21 The difference, however, is that we need to minimize the 
satisfaction probability. The latter can be achieved by a very similar SSAT encoding scheme 
that exploits universal quantifiers to resolve non-deterministic transition choices. Universal 
quantifiers then aim at minimizing the satisfaction probability. Though the SSMT solver 
SiSAT actually supports universal quantification, confer [TF09[ ITEFllj . we instead stay 
within the scope of the logic exposed in this article and rephrase minimum probabilistic 
state reachability as a maximum probabilistic state avoidance problem as follows: 

;s G JC 

1 ;s<£JC,k = 
max Yl P s i s , a i s ') ' MaxAvoid^^s') ; s £ JC, k > 



MaxAvoidj^^s) 
It then holds that 



a&Act s , gS 



MinReach k M K {i) = 1 - MaxAvoid k MK (i) 



which can be proven by straightforward induction over step bound k. In the base cases, i.e. 
if k = and s G JC or s ^ JC, the statement is clear. Within the induction step, we exploit 
the property that 

mini ^2 Pi j ■ Pi,j = 1 - max; ^ p itj • (1 - P itj ) 

is true for < Pij < 1 and YljPiJ = !• 

The problem of computing the value of MaxAvoid k M ^(i) can be reduced to computing 
the maximum probability of satisfaction of the SSAT formula 

states reachable within k steps avoid invariance kernel 

®MK = G(*0 : ( Init(so) A A. Trans(si-i,ti, S{) A A. ^Kfa) 

According to the definition of MaxAvoid k M ^{i), the propositional formula of <& k M ^ describes 
all system runs avoiding the invariance kernel JC for at least k transition steps. That is, all 
assignments encoding such latter runs yield satisfaction probability 1, while assignments en- 
coding runs that visit JC within the first k steps do not satisfy the propositional formula, thus 
leading to satisfaction probability 0. As a consequence, MaxAvoid k M ^(i) = Pr y^MfA- 
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Table 2: Experimental results of applying the generalized interpolation scheme I4.6I on Ad 
from Figure [5] for different values of parameter j. In addition to the symbolic 
representations computed by interpolation, the concrete state sets represented by 
these predicates are stated explicitly. 

Using above facts, we deduce the following relation 

1 - Pr (®m jK ) = 1 - MaxAvoid k M K {i) 

= MinReach k M 

< MinReach k MK *(i) 

< MinStable(M, Region) . 

This finally enables us to compute lower bounds Ib^ of MinStable(Ad, Region) using the 
scheme 

lb k :=l-Pr($ k MiK ) , (4.7) 

the latter being addressed by SSAT solving. Note that the system behavior encoded by 
K, becomes more and more constrained for increasing k such that the satisfaction prob- 
abilities Pr fcj are monotonically decreasing. This in turn means that the Ib^s are 
monotonically increasing. With regard to solving the stability verification problem 14.51 the 
desired property MinStable(A4, Region) > 6 is verified by the procedure above once a lower 
bound Ibk > 6 is computed for some k. 

Example. To illustrate the symbolic approach to probabilistic region stability based on 
generalized Craig interpolation, again consider the simple MDP Ai from Figure 0] where 
the symbolic representation of the region is given by Region(s) = —if. That is, the region 
in which A4 should stabilize consists of the states i, e, and s. The symbolic SSAT encoding 
of Ad being introduced in the example of Section 14.11 is reused in the following. 

We are first interested in computing an invariance kernel /C C Region(s) with respect 
to Ad by means of the generalized Craig interpolation scheme 14.61 To cope with the latter 
scheme automatically, we employ the simple interpolating DPLL-based SSAT solver men- 
tioned in Section 14.11 The results of these experiments for different values of j are shown in 
Table [2j It is not hard to see that the unique maximal invariance kernel KL* consists of the 
state s only. Recall that each interpolant X k+1 overapproximates all system states directly 
leading to the state set —iJZ k . When setting parameter j to value 1 or 2, we observe that 
interpolant I 1 = true is too coarse since it includes the whole state space. This causes the 
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V K° = Region 2 4 6 8 10 

~ - - number k of transition steps 

Figure 6: Illustration of the computed state sets for MDP A4 by the generalized interpo- 
lation scheme [4.6I with j € {3,4} (left), and lower bounds Ibk of the minimum 
probability of reaching the invariance kernel fC = {s} over number k of transition 
steps computed by scheme l4"77l (right). 



trivial invariance kernel K, = false representing the empty set. For choices j = 3 and j = 4, 
however, Z 1 = -is describes the exact set of states which lead to -<1Z = — 'Region. Finally, 
the non-trivial invariance kernel /C = —>f A s consisting of state s only is computed. Note 
that fC actually is the maximal invariance kernel. The computed state sets for j E {3, 4} 
are illustrated on the left of Figure [6j 

These results confirm the observation made from the experiments of Section \A.\\ namely 
that the greater the value of j, i.e. the more transition steps are performed, the more 
accurate the resulting overapproximations. Concerning runtime, each generalized Craig 
interpolant was computed by the interpolating DPLL-based SSAT solver within fractions 
of a second, where the highest runtime of 88 milliseconds was observed when computing I 2 
for j = 4. 

Having computed an invariance kernel fC{s) C Region(s) with respect to A4, we are now 
able to compute lower bounds Ibk of the minimum probability that Ad is stable with respect 
to Region by means of scheme I4T71 where we use JC(s) = ->f A s as obtained for j £ {3,4}. 
Employing the SSMT tool SiSAT, some of the results are Ibo = lb\ = 0, Z62 = ^3 = 0.45, 
Z64 = W5 = 0.54, . . ., Z&100 = 0.54. Concerning runtime, all 100 SSAT formulae were solved 
within 88.16 seconds, while computation of the first 20 lower bounds Ibo to Z&20 just needed 
600 milliseconds. The highest computation time for a single SSAT problem was obtained for 
Ibioo, namely 2.91 seconds. The evolution of the Ib^s up to k = 10 is presented graphically 
on the right of Figure El With regard to the stability verification problem 14.51 the desired 
property MinStable(A4, Region) > is verified for each threshold value 6 < 0.54 within a 
second. 

Concerning competitive approaches, we remark that the probabilistic model checking 
tool PRISM 4.0.1 [KNP11] is also able to deal with probabilistic region stability of MDPs 
by means of path operators^ To determine the value of MinStable(A4, Region) for the 
example above, we used the specification Pmin=? [FP>=1 [G (!f)]] meaning that we are 
interested in the minimum probability (Pmin=?) that finally (F) the system satisfies almost 

^Confer http: //www.prismmodelchecker . org/manual/PropertySpecif ication/ThePOperator for 

more detailed information. 
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surely (P>=1) the property that globally (G) state / is never visited (!f). PRISM solved 
the problem in 644 milliseconds returning the result 0.54. 

As discussed for the case of probabilistic state reachability at the end of Section 14.11 
we are also confident that the presented approach to probabilistic region stability based 
on generalized Craig interpolation becomes beneficial when adapted to probabilistic hybrid 
systems, where the classical procedures are not directly applicable. Furthermore, a partic- 
ular pay-off is expected when dealing with concurrent probabilistic systems owing to the 
symbolic nature of the interpolation-based technique. 

5. Conclusion and future work 

In this article, we elaborated on the idea of Craig interpolation for stochastic Boolean sat- 
isfiability. In consideration of the difficulties that arise in this stochastic extension of the 
propositional satisfiability problem, we first proposed a suitable definition of a generalized 
Craig interpolant and then presented an algorithm for automatically computing such inter- 
polants. For the latter purpose, we enhanced the SSAT resolution calculus by corresponding 
rules for the construction of generalized Craig interpolants. We furthermore demonstrated 
two applications of generalized Craig interpolation as a means of automated analysis of 
probabilistic finite-state systems. 

We first considered probabilistic state reachability. The resulting procedure is able to 
verify probabilistic safety requirements of the form "the worst-case probability of reach- 
ing undesirable system states is at most some given safety threshold". This complements 
the existing SSAT-based probabilistic bounded model checking approach, which mechanizes 
falsification of such safety properties. As a second application, we gave attention to proba- 
bilistic region stability and presented a symbolic technique for verifying stability properties 
like "the worst-case probability that the system stabilizes within some given region is at 
least some given safety threshold". 

For future work, we are particularly interested in the adaptation of generalized Craig 
interpolation to SSMT, i.e. the extension of SSAT with arithmetic theories. One of the 
most challenging issues here will be the enhancement of the SSAT resolution calculus as 
well as the corresponding rules for the construction of generalized interpolants in order to 
deal with SSMT problems. The ability of computing generalized Craig interpolants for 
SSMT would lift the interpolation schemes 14.31 and 14.61 to SSMT problems, thus establish- 
ing symbolic verification approaches to probabilistic state reachability and to probabilistic 
region stability for discrete-time probabilistic hybrid systems. We are confident that such 
symbolic procedures will prove beneficial within the analysis of probabilistic hybrid systems, 
in particular when systems with a high degree of concurrency are considered. 
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